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ABSTRACT 

Cognitive Academic Language Learning Approach 
(Project CALLA) was a federally funded program serving 960 
limited-English-proficient students in 10 Manhattan (New York) 
elementary schools in 1992-93 its third year of operation. The 
project provided instruction in English as a Second Language (ESL) , 
mathematics, science, and social studies in kindergarten through 
grade 6. Staff development workshops, individualized staff training, 
parent ESL instruction, and parent involvement activities were also 
important program components. The project met all its stated 
objectives in English language development and the content areas, 
staff development, and the parent involvement objective for 
understanding children ! s education, but not the parent involvement 
objective for parent participation in activities. It is suggested 
that in this case, either attendance records were inadequate or the 
participation level targeted was unrealistic. The major 
recommendation for program improvement was to modify this objective. 
Appended materials include lists of instructional materials, class 
schedule, and staff and parent questionnaires used in program 
eva 1 ua t i on . (MSE) 
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EXECUTIVE SUMMARY 



The Cognitive Academic Language Learning Approach (Project CALLA), an 
Elementary and Secondary Education Act (E.S.E.A.) Title Vli-funded project, was in its 
third year in 1 992-93. It functioned at ten schools in Community School District 
(C.S.D.) 2 in Manhattan: P.S. 1, 2, 33, 42, 51, 111, 116, 124, 151, and 198. In the 
year under review, Project CALLA served 960 limited English proficient (LEP) 
students. LEP status was determined by scores at or below the 40th percentile on 
the Language Assessment Battery (LAB). 

Staff development was an important focus of Project CALLA. Participating 
teachers attended workshops throughout the year. They also received individualized 
training to meet the special needs of LEP students, with emphasis on improving their 
English reading and content area skills. Parents were offered E.S.L. classes and the 
opportunity to attend a variety of meetings throughout the year. 

The project met all of its student objectives in English language development 
and the content areas. It also met all staff development objectives. The project met 
the parental involvement objective for understanding children's education but did not 
meet the objective for parental participation in activities. This may have been due to 
inadequate attendance records, or the fact that the number of parents projected to 
participate (60 percent) was unrealistic. The project director indicated that greater 
emphasis will be placed on documenting the involvement of project parents in 
activities. 

The following recommendation was made based on the findings of this 
evaluation: 

• Seek to modify the objective for parental participation in activities, to 
make it more realistic. 
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I. INTRODUCTION 

This report documents the Office of Research, Evaluation, and Assessment's 
(OREAs) 1992-93 evaluation of the Elementary and Secondary Education Act 
(E.S.E.A.) Title VII program, Cognitive Academic Language Learning Approach 
(Project CALLA). 

PROJECT CONTEXT 

The project was in its third year at P.S. 1 , 2, 33, 42, 51 , 1 1 1 , 1 1 6, 1 24, 1 51 , 

and 198 in Community School District (C.S.D.) 2 in Manhattan. 

Of the 605 students at P.S. 1, 72 percent (433) were Asian-American, 14 
percent (87) were Latino, 13 percent (76) were African-American, and 1.0 percent (9) 
were European-American. Ninety-one percent of the students were eligible for the 
federal free-lunch program, an indicator of low family income, and 37 percent of 
students were of limited English proficiency (LEP). 

Of the 882 students at P.S. 2, 81 percent (714) were Asian-American, 10 
percent (84) were Latino, 7 percent (65) were African-American, and 2 percent (19) 
were European-American. Ninety percent of the students were eligible for free lunch, 

and 34 percent were LEP. 

Of the 340 students at P.S. 33, 58 percent (197) were Latino, 22 percent (75) 
were African -American, 12 percent (41) were Asian-American, and 8 percent (27) 
were European-American. Eighty-four percent of students were eligible for free lunch 

and 29 percent were LEP. 

Of the 871 students at P.S. 42, 83 percent (721) were Asian, 13 percent (113) 
were Latino, 3 percent (26) were African-American, and 1 percent (11) was European- 



American. Ninety-four percent of students were eligible for free lunch and 45 percent 
of the students were LEP. 

Of the 257 students at P.S. 51, 60 percent (154) were Latino, 24 percent (62) 
were African-American, 11 percent (27) were European-American, and 5 percent (14) 
were Asian-American. Ninety-three percent of students were eligible for free lunch 
and 30 percent were LEP. 

Of the 633 students at P.S. 111, 71 percent (448) were Latino, 14 percent (91) 
were African -American, 11 percent (69) were European-American, and 4 percent (25) 
were Asian-American. Eighty-eight percent of students were eligible for free lunch 
and 22 percent were LEP. 

Of the 687 students at P.S. 1 16, 33 percent (229) were Latino, 30 percent (204) 
were African -American, 25 percent (175) were European-American, and 12 percent 
(79) were Asian-American. Sixty-four percent of students were eligible for free lunch, 
14 percent were LEP. 

Of the 1,1 13 students at P.S. 124, 92 percent of students (1,033) were Asian- 
American, fou- percent (41) were European-American, two percent (22) were African- 
American, and two percent (22) were Latino. Seventy -three percent of students were 
eligible for free lunch, and 28 percent were LEP. 

Of the 247 students at P.S. 151, 35 percent (87) were Latino, 31 percent (76) 
were African-American, 26 percent (64) were European-American, and 8 percent (20) 
were Asian-American. Eighty-one percent of the students were eligible for free lunch, 
and 14 percent of students were LEP. 

Of the 367 students at P.S. 198, 57 percent (210) were Latino, 27 percent (100) 
were African-American, 11 percent (38) were European-American, and 5 percent (19) 
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were Asian-American. Seventy-eight percent of the students were eligible for free 
lunch, and 19 percent were LEP. 

STUDENT CHARACTERISTICS 

Project CALLA served a total of 960 LEP students (see Table 1), the majority of 
whom were Spanish- or Chinese-speaking students in kindergarten through sixth 
grade. Project students came from a variety of countries. (See Table 3.) Students 
who scored at or below the 40th percentile on the Language Assessment Battery 
(LAB) were determined to be LEP. Male students numbered 491 (51 percent) and 
female 467 (49 percent); gender was not reported for two. Approximately 87 percent 
of project participants came from low-income families and were eligible for the free- 
lunch program. 

TABLE 1 

Number of Students in Project CALLA, by Site 



Site 


Total 


P.S. 1 


47 


P.S. 2 


109 


P.S. 33 


62 


P.S. 42 


142 


P.S. 51 


83 


P.S. 111 


66 


P.S. 116 


132 


P.S. 124 


200 


P.S. 151 


32 


P.S. 198 


87 


Total 


960 
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TABLE 2 
Students' Native Languages 



Native Language 


Number of Students 


Spanish 


271 


Chinese 


457 


Cantonese 


371 


Chinese (Dialect Unknown) 


33 


Fukinese 


Q1 
O i 


Mandarin 


22 


English 


99 


Arabic 


28 


Romanian 


12 


Not Reported 


93 



Needs Assessment 

Before instituting Project CALLA, C.S.D. 2 conducted a complete needs 
assessment. Using the results of the LAB, it was determined that in the 1989-90 
school year, the C.S.D. had 2,077 Chinese-speaking, 569 Spanish-speaking, and 247 
students speaking a variety of other languages who were LEP and were not receiving 
services. 

To familiarize them with English and help them acculturate, Project CALLA 
proposed to provide services to these students. 
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TABLE 3 



Students' Countries of Origin 



Country 


Number of Students 


Unitec States 


486 


China 


90 


Hong Kong 


25 


Dominican Republic 


21 


Yemen 


18 


Mexico 


12 


Yugoslavia 


11 


Philippines 


11 


Ecuador 


10 


Romania 


10 


Bangladesh 


5 


Bhutan 


5 


Korea 


5 


Pakistan 


4 


Poland 


4 


Venezuela 


A 
H 


Colombia 


3 


Malaysia 


3 


Nambia 


3 


Not Reported 


200 


Other 


30 
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PROJECT OBJECTIVES 



Student Objectives 

• By June 30, 1993, 60 percent of participating students will improve their 
English language skills as demonstrated by a significant increase in 
scores on the LAB, as measured by a correlated t-test for significance of 
difference between pre- and post-test scores converted into N.C.E.s. 

• By June 30, 1993, at least 50 percent of a random, stratified sample of 
the target population who began at the beginning or intermediate level 
of English language academic material linguistic difficulty, and who have 
been receiving project services for at least five months, will have 
progressed at least one level (from beginning to intermediate, from 
intermediate to advanced) in the difficulty of material they are using in 
their subject area classes, as measured by reviews of participating 
teacher class records by District E.S.L. staff. 

• By June 30, 1993, at least 50 percent of a random, stratified sample of 
the target population will show improvement in math, science, and 
social studies by maintaining a satisfactory or improving one level, as 
demonstrated by a comparison of the November report card to the end 
of year report card. 

Staff Development Objectives 

• By November 30, 1992, the Resource Teacher will have completed 
classroom-specific needs assessments of LEP students and teachers of 
LEP students in each target school, as measured by needs assessment 
logs. 

• By June 30, 1993, the Resource Teacher will have successfully provided 
a range of project services to LEP students, as measured by project 
logs. 

• By March 30, 1993, the Resource Teacher will have collected, modified, 
and developed a library of E.S.L. resources for the teachers in the target 
schools, as measured by the existence and use of the library. 

• By June 30, 1993, the Resource Teacher will have successfully provided 
a range of project services to the teachers of LEP students, as"' 
measured by project logs and independent observations of the 
techniques used by teachers of LEP students in their classrooms. 
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• Bv June 30 1993, the Resource Teacher will have begun to collect, edit, 
and distribute the best of the teacher-developed units in each subject 
area, as measured by the creation and distribution of such materials. 

• Bv September 1 , 1992, 35 teachers at the target schools will have 
completed a one-week summer institute on CALLA techniques and 
methodologies. 

• Bv September 1 , 1992, 80 percent of the participating teachers in the 
summer Sute will have demonstrated knowledge of the content and 
meThodology of cognitive approaches to English language development, 
as measured by pre- and post-training questionnaires. 

• Bv February 1, 1993, all participating teachers will have completed 
fndivfdual needs assessments for working with their LEP students, as 
measured by the Resource Teacher needs assessment logs. 

• 3 V June 30 1993, 90 percent of participating teachers will have received 
direct, individualized services from the Resource Teacher. 

• Bv June 30, 1993, 90 percent of participating teachers will demonstrate 
the use of project-provided E.S.L. strategies, as well as modify and 
develop their own materials and/or use materials modified and 
developed by their colleagues to better meet the needs of the LEP 
students as measured by a questionnaire. 

• By June 30, 1 993, 65 percent of the participa ting teacher s will have 
established mechanisms for meeting regularly with E.S.L/b.l.ngual 
and/or subject area teachers of LEP students at the same grade level, 
as measured by participating teacher and Resource Teacher logs. 

Parental Involvement Objectives 

• Bv June 1993, at least 60 percent of the parents of participating 
students will have taken part in three or more activities for parents, as 
measured by attendance records for parental involvement activities. 

• Bv June 1 993, at least 60 percent of the parents of participating 
students will have demonstrated a satisfactory level of understandmg o 
and participation in their children's education, as measured by a parent 
checklist and by teachers reports. 
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PROJECT IMPLEMENTATION 

During the 1992-93 school year, Project CALLA provided instructional and 
support service to 960 LEP students and their families. The main goals of the project 
were to improve LEP students' English language reading and reading-related 
achievement skills; increase their science, social studies, and mathematics skills; and 
provide training to better enable teachers to meet the needs of LEP students. 

In order to attain these goals, the project conducted extensive staff 
development throughout the year and at a summer institute. Teachers were trained 
to use the cognitive, academic, language learning approach (CALLA) which 
addresses the particular academic vocabulary and linguistic concepts that students 
must master if they are to succeed in academic subject areas. 
Capacity Building 

The project intends to use tax-levy or Part 154 funding to provide coverage for 
target staff who will be released for training. 
Materials, Methods, and Techniques 

All content area courses were taught with an E.S.L. methodology. Instructional 
strategies used for E.S.L. and the content areas included whole language total 
physical response, hands-on activities, cooperative learning, peer tutoring, and the 
CALLA approach. 

For a list of instructional materials used, see Appendix A. 
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Staff Qualifications 

Title Vll st aff. The project's Title VII staff consisted of an E.S.L. resource 
specialist, who also served as project coordinator, and a school neighborhood 
worker. For a description of degrees held, and language competencies, see T 



TABLE 4 
Project Staff Qualifications 



Position Title 


Degree(s) 


Language 
Competence 


Resource 

Specialist/Project 

Coordinator 


M.A. 


Spanish 


School Neighborhood 
Worker 


H.S. 


Chinese 



The responsibilities of the E.S.L. resource specialist/project coordinator 
included providing teacher training, parent workshops, and E.S.L classes; ordering 
and disseminating materials; maintaining the resource center at P.S. 198; and 

districtwide training. 

The responsibilities of the neighborhood worker included dissemination of 
materials, translation of materials for Chinese parents, accompanying parents to 
conferences, and providing administrative support. 
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Other staff . Tax-levy funds paid the salaries of ten classroom teachers. For a 
list of degrees held, certification, and language competence (teaching or 
communicative proficiency*), see Table 5. All staff were certified in the area they 
taught. 

TABLE 5 
Qualifications of Non-Title VII Staff 



Title 


Degree(s) 


Certificates/Licenses 


Language 
Competence 


10 Teachers 


10 MA 


7 E.S.L 

1 Common 
Branches 

2 C.B./E.S.L. 


2 Spanish TP 
2 French TP 
1 Chinese TP 



Staff development . Teacher training was an integral part of Project CALLA. 
Workshops were given throughout the year, cove.mg such topics as CALLA 
approaches, E.S.L. strategies in content areas, hands-on activities in the sciences, 
and cooperative learning in the content areas. CALLA teachers attended at least four 
full-day training sessions and had the opportunity to visit a "master teacher" or 
exemplary program. The resource specialist met with individual teachers once or 
twice a week to offer assistance. In addition, the resource specialist provided 
numerous model iessons for participating students and teachers and followed up with 
meetings and visits to the classroom. Teachers also received a week of training 

*Teaching proficiency (TP) is defined as the ability to use LEP students' native 
language in teaching language arts or other academic subjects. Communicative 
proficiency (CP) is defined as a non-native speaker's basic ability to communicate 
and interact with students in their native language. 
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during the summer institute. 

The district put out an E.S.L/Bilingual newsletter, "The Global Class," which 
shared E.S.L. strategies and CALLA approaches and ofrered information on meetings, 
conferences, and events of interest to E.S.L., bilingual, and project teachers. 
Instructional Time Spent on Particu lar Tasks 

See Appendix B for an example of a class schedule. 
Length of Time Participants Receiv ed Instruction 

Students had a median of 1.16 years of education in a non-English-speaking 
school system and 1 .92 years of education in the United States. Average 
participation in Project CALLA was 9.45 months. 

Activities to Improve Pre-referral Evaluati on Procedures for Exceptional Students 

Students who were deemed gifted and talented were encouraged by Project 
CALLA to participate in special programs, such as plays and exhibitions. Students 
who were physically challenged were given individual lessons with modified activities 
as necessary. Project students also received special services such as reading 
recovery, the facilities of the resource room, speech therapy, counseling, mentoring, 
and remedial mathematics as required. 

Students were referred to special education because of low academic 
performance, low standardized test scores, emotional/behavioral problems, or speech 
disorders. All schools had School-Based Support Team (S.B.S.T.) members that 
included bilingual social workers and educational evaluators, or other staff members 
who were fluent in the language of the students. 
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PARENT AND COMMUNITY INVOLVEMENT ACTIVITIES 

Parents were provided with E.S.L classes two days a week at P.S. 33, P.S. 
111, and P.S. 198. They attended Parent Advisory Committee (PAC) meetings 
throughout the year. Parents attended the New York State Association for Bilingual 
Education (SABE) Parents Institute as well as the National Association for Asian and 
Pacific-American Education (NAAPAE) Parents Institute. 

Project CALLA published a parent newsletter in English/Spanish and 
English/Chinese that was distributed to ali parents. 
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II. EVALUATION METHODOLOGY 



FVALUATION DESIGN 

Project Group's Educational Progress as Compared to That of an Appropriate 
Non-Project Group 

OREA used a gap reduction design to evaluate the effect of language 
instruction on project students's performance on standardized tests. Because of the 
difficulty in finding a valid comparison group, OREA used instead the groups on 
which the tests were normed. Test scores are reported in Normal Curve Equivalents 
(N.C.E.s), which are normalized standard scores with a mean of 50 and a standard 
deviation of 21.1. It is assumed that the norm group has a zero gain in N.C.E.s in 
the absence of supplementary instruction and that participating students' gains are 
attributable to project services. 

Applicability of Conclusions to All Persons Se rved bv Project 

Data were collected from all participating students for whom there were pre- 
and posttest scores. (There were no pretest data on students who entered the 
program late; therefore, posttest data for them will serve as pretest data for the 
following year.) Instruments used to measure educational progress were appropriate 
for the students involved. The LAB is used throughout New York City to assess the 
growth of English skills in populations similar to those served by Project CALLA. 

INSTRUMENTS OF MEASUREMENT 

OREA compared pre- and posttest scores on the LAB to assess the first E.S.L. 
objective. The second E.S.L objective, which called for progression of a random, 
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stratified sample by one level, was measured by examining the students level at the 
start of the year and at the end. The content area objective in mathematics, science, 
and social studies was assessed through course grades, as specified. 

All students were tested at the appropriate grade level. The language of the 
LAB was determined by the test itself. 

According to the publishers' test manual, the LAB is valid and reliable. 
Evidence supporting both content and construct validity is available for the LAB. 
Content validity is confirmed by an item-objective match and includes grade-by-grade 
item difficulties, correlations between subtests, and the relationship between the 
performance of students who are native speakers of English and students who are 
LEP. To support reliability, the Kuder-Richardson Formula 20 (KR20) coefficients and 
standard errors of measurement (SEM) are reported by grade and by form for each 
subtest and total test. Grade reliability coefficients, based on the performance of LEP 
students on the English version, ranged from .88 to .96 for individual subtests and 
from .95 to .98 for the total test. 

To assess the objectives for staff training, the project resource teacher 
reported to OREA. OREA also developed and distributed evaluation questionnaires 
for teachers to report on their knowledge of the content and methodology of cognitive 
approaches to English language development, and the use of project provided E.S.L 
strategies and the modification and development of materials. (See Appendix C.) 
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in order to assess the parental involvement objectives, OREA developed and 
used a questionnaire translated into Chines r and Spanish. (See Appendix C.) The 
second parental involvement objective was measured by attendance records. 

DATA COLLECTION AND ANALYSIS 
Data Collection 

To gather qualitative data, an OREA evaluation consultant carried out on-site 
and telephone interviews with the project director several times during the school 
year and also observed two classes on each of two visits. The project evaluator 
collected the data and prepared the final evaluation report in accordance with the 
New York State E.S.E.A. Title VII Bilingual Education Final Evaluation Report format, 
which was adapted from a checklist developed by the staff of the Evaluation 
Assistance Center (EAC) East in consultation with the Office of Bilingual Education 
and Minority Language Affairs (OBEMLA). 
Pr oper Administration of Instruments 

Qualified personnel received training in testing procedures and administered 
the tests. Test administrators followed guidelines set forth in the manuals 
accompanying standardized tests. Time limits for subtests, were adhered to; 
directions were given exactly as presented in the manual. 
Testing at Twelve-month Testing Intervals 

Standardized tests were given at 12-month intervals, following published 
norming dates. 
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Data Analysis 

Accurate scoring and transcription oLre sults. Scoring, score conversions, and 
data processing were accomplished electronically by the Scan Center of the Board of 
Education of the City of New York. Data provided by the Scan Center were analyzed 
in the Bilingual, Multicultural, and Early Childhood Evaluation Unit of OREA. Data 
collectors, processors, and analysts were unbiased and had no vested interest in the 

success of the project. 

Use of analyses and reporting procedures a p pro p ri ate f or obtained data . To 
assess the significance of students' achievement in English. OREA computed a 
correlated f-test on LAB N.C.E. scores. The f-test determined whether the difference 
between the pre- and posttest scores was significantly greater than would be 
expected from chance variation alone. 

The only possible threat to the validity of the above instrument might be that 
LAB norms were based on the performance of English Proficient (EP) rather than LEP 
students. Since OREA was examining gains, however, this threat was 
inconsequential-the choice of norming groups should not affect the existence of 
gains. 
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III. FINDINGS 

PARTICIPANTS' EDUCATIONAL PROGRESS 

E.S.L classes were offered four to five times per week. Instructional strategies 
used for E.S.L included total physical response, whole language, the CALLA 
approach, cooperative learning, language experience, and peer tutoring. 
LEP Participants' Progress in English 

An OREA evaluation consultant observed a kindergarten E.S.L. class at P.S. 
116. The children were working on their daily journals and wrote a sentence or two 
to accompany a picture that they drew. The topic of the day was what the children 
intended to do on their spring vacation. If the children had trouble with a word, the 
teacher would help them sound it out. In some cases, the teacher would ask them to 
find the word written somewhere in the room as a guide. The students' work was 
displayed around the room. 

The OREA consultant also visited a first grade E.S.L. pull-out class at P.S. 116. 
The class was for new immigrants, ages seven through thirteen. The teacher went 
over the calendar with the students and also gave them their own calendar to fill in. 
The students listened to the tape "Jazz Chants for Children," reading along from a 
book. The teacher gave out cards with phrases from the tape, and each child read a 
phrase. The children then played a matching game with their cards. 
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The evaluation objective for English language development was: 

• By June 30, 1993, 60 percent of participating students will improve their 
English language skills as demonstrated by a significant increase in 
scores on the LAB, as measured by a correlated t-test for significance of 
difference between pre- and post-test scores converted into N.C.E.s. 

There were complete pre- and posttest scores on the LAB for 867 students in 
kindergarten through sixth grade. (See Table 6.) An average of 76 percent of 
students demonstrated a significant increase. Students showed a mean gain of 17.8 
N.C.E.s (s.d.= 22.4), which was statistically significant (p<.05). 

The project met its objective for English language development. 



• By June 30, 1993, at least 50 percent of a random, stratified sample of 
the target population who began at the beginning or intermediate level 
of English language academic material linguistic difficulty, and who have 
been receiving project services for at least five months, will have 
progressed at least one level (from beginning to intermediate, from 
intermediate to advanced) in the difficulty of material they are using in 
their subject area classes, as measured by reviews of participating 
teacher class records by District E.S.L. staff. 

Ninety-five students were selected as a random, stratified sample of the target 
population. Of those students for whom data were reported, 69 percent progressed 
at least one level of E.S.L by the end of the project year. (See Table 7.) 

The project met its objective for E.S.L 
LEP Participant's Progress in Native Language 

Since this was a special alternative project serving many language groups, 
native language instruction was not offered. 
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LEP Participant's Academic Achievement 

Instructional strategies used for the content areas included the CALLA 

approach, cooperative learning, language experience, hands-on mathematics and 

science, whole language, and peer tutoring. 

• By June 30, 1993, at least 50 percent of a random, stratified sample of 
the target population will show improvement in math, science, and 
social studies by maintaining a satisfactory or improving one level, as 
demonstrated by a comparison of the November report card to the end 
of year report card. 

Based on the random, stratified sample chosen, 80 percent of students 
maintained a satisfactory level or showed improvement in mathematics (see Table 7) 
90 percent maintained a satisfactory level or showed improvement in science (see 
Table 8); and 88 percent maintained a satisfactory level or showed improvement in 
social studies (see Table 8). 

The project met its objective for content areas. 
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TABLE 7 



Final Grades in 
E.S.L. and Mathematics, by Site 

Fall 1992/Spring 1993 





E.S.L. 


MATHEMATICS 


Site 


Number of 
students for 
whom data 
were reported 


Percentage of 

students 
increasing at 
least one level 


Number of 
students for 
whom data 
were reported 


Percentage of 

students 
maintaining a 
satisfactory level 
or improving 


P.S. 1 


5 


100 


5 


100 


P.S. 2 


9 


88 


9 


77 


P.S. 33 


9 


78 


9 


QQ 
OO 


P.S. 42 


9 


77 


9 


44 


P.S. 51 


10 


30 


10 


80 


P.S. 111 


10 


100 


10 


100 


P.S. 116 


8 


100 


10 


100 


P.S. 124 


10 


50 


10 


100 


P.S. 151 


10 


70 


9 


66 


P.S. 198 


10 


20 


10 


50 


Total 


90 


69 


91 


80 
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Table 8 

Final Grades in 
Science and Social Studies, by Site 

Fall 1992/Spring 1993 





Science 


Social Studies 




Number of 
students for 
whom data 
were reported 


Percentage of 

students 
maintaining a 
satisfactory level 
or improving 


Number of 
students for 
whom data 
were reported 


Percentage of 

students 
maintaining a 
satisfactory level 
or improving 


P.S. 1 


5 


100 


5 


100 


P.S. 2 


9 


100 


9 


100 


P.S. 33 


rv 

y 


— — 


q 


100 


P.S. 42 


9 


88 


9 


67 


P.S. 51 


10 


80 


10 


80 


P.S. 111 


10 


100 


10 


100 


P.S. 116 


10 


100 


10 


90 


P.S. 124 


10 


100 


10 


90 


P.S. 151 


9 


77 


7 


86 


P.S. 198 


10 


80 


10 


70 


Total 


91 


90 


89 


88 
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FORMER PARTICIPANTS' PROGRESS IN ENGLISH LANGUAGE C LASSROOMS 

Four project students were mainstreamed following the 1991-92 school year. 
Their subsequent progress was not monitored. 

OVERALL EDUCATIONAL PROGRESS ACHIEVED THROUGH PROJECT 
Mainstreaminq 

This year 47 students (5 percent) were mainstreamed. 

Grade Retention 

Last year, one project student was retained in grade. This year, no students 
were retained. Project CALLA did not propose any objectives for grade retention. 
Attendance 

Project CALLA did not propose any objectives for attendance. The overall 
project attendance rate for the year under review was 91.7, based on 960 students. 
(See Table 9.) 

Placement in Gifted and Talented Programs 

No project students were referred to gifted and talented programs following the 
1991-92 school year. This year, 11 students were placed in gifted and talented 
programs. 
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TABLE 9 

Project and Mainstream Students' Attendance, by Site 



Site 


Project Student Attendance 


Mainstream Attendance 


P.S. 1 


96.0 


93.1 


P.S. 2 


93.4 


90.4 


P.S. 33 


86.0 


84.1 


P.S. 42 


94.4 


93.5 


r.o. oi 


86 8 


92.5 


P.S. 111 


90.3 


88.2 


P.S. 116 


87.0 


87.7 


P.S. 124 


97.1 


96.6 


P.S. 151 


88.5 


88.4 


P.S. 198 


88.9 


77.5 



Case History 

i.A. was an eight-year-old second grader from Ecuador who had lived in the 
United States for three years. I.A.'s parents did not speak English and reported that 
they were not able to help him with his homework. Despite this, I.A. made good 
progress in school. He scored at tha 99th percentile on the spring 1993 LAB and 
received grades of "good" and above from his classroom teacher. His attendance at 
school was also very good. I.A.'s participation in Project CALLA contributed to his 
success in school, as did his mother's increased involvement in his education— she 
attended a number of parent meetings and workshops. 
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STAFF DEVELOPMENT OUTCOMES 

• By November 30, 1992, the Resource Teacher will have completed 
classroom-specific needs assessments of LEP students and teachers of 
LEP students in each target school, as measured by needs assessment 
logs. 

The resource specialist administered a questionnaire in the fall of 1992 to staff 
both at workshops and individually, that addressed staff goals and needs for 
development activities in E.S.L. and E.S.L in the content areas. 

The project met its objective for completion of needs assessment logs. 

• By June 30, 1993, the Resource Teacher will have successfully provided 
a range of project services to LEP students, as measured by project 
logs. 

The resource teacher provided numerous model lessons to participating 
students and teachers, following up with meetings and visits to the classrooms. The 
resource teacher worked with students from P.S. 111 on a Puerto Rican Heritage Fair 
project that won third place in a citywide competition. The class was invited to a 
special luncheon in honor of the winners. The resource teacher also worked on a 
play, The Music Machine, at P.S. 198. Two performances were given for staff, 
parents, and students. The project coordinator maintained a log which documented 
these activities. 

The project met its objective for the resource teacher providing services to LEP 
students. 
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• By March 30, 1993, the Resource Teacher will have collected, modified, 
and developed a library of E.S.L resources for the teachers in the target 
schools, as measured by the existence and use of the library. 

The resource teacher maintained a library of E.S.L. resources she established 
at P.S. 198. 

The project met its objective for an E.S.L. resource library. 

• By June 30, 1993, the Resource Teacher will have successfully provided 
a range of project services to the teachers of LEP students, as 
measured by project logs and independent observations of the 
techniques used by teachers of LEP students in their classrooms. 

The resource teacher provided E.S.L training, and individualized follow-up to 
teachers of LEP students. 

The project met its objective for providing training to teachers. 

• By June 30, 1993, the Resource Teacher will have begun to collect, edit, 
and distribute the best of the teacher-developed units in each subject 
area, as measured by the creation and distribution of such materials. 

Samples from a collection of teacher-developed lessons/units were updated 

throughout the year and were distributed to teachers in June of 1993. 

The project met its objective for creation and distribution of teacher-developed 

units. 

• By September 1 , 1992, 35 teachers at the target schools will have 
completed a one-week summer institute on CALLA techniques and 
methodologies. 

Thirty-five teachers at the target Project CALLA schools completed a one-week 
summer institute on CALLA techniques and methodologies. The project met its 
objective for teacher attendance at the summer institute. 
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• By September 1, 1992, 80 percent of the participating teachers in the 
summer institute will have demonstrated knowledge of the content and 
methodology of cognitive approaches to English language development, 
as measured by pre- and post-training questionnaires. 

OREA provided a questionnaire for teachers to indicate whether the summer 
institute increased their knowledge of the content and methodology of cognitive 
approaches to English language development. All questionnaires were returned and 
all indicated that respondents had increased their knowledge as a result of 
participating in the summer institute. 

The project met its objective for staff increase of knowledge. 



• By February 1, 1993, all participating teachers will have completed 
individual needs assessments for working with their LEP students, as 
measured by the Resource Teacher needs assessment logs. 

All teachers completed needs assessments on individual students and updated 
them throughout the year. 

The project met its objective for teacher completion of needs assessments. 



• By June 30, 1993, 90 percent of participating teachers will have received 
direct, individualized services from the Resource Teacher. 

The resource specialist met with participating teachers and gave individualized 
training throughout the year. 

The project met its objective for teachers receiving services from the resource 

teacher. 
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• By June 30, 1993, 90 percent of participating teachers will demonstrate 
the use of project-provided E.S.L strategies, as well as modify and 
develop their own materials and/or used materials modified and 
developed by their colleagues to better meet the needs of the LEP 
students as measured by a questionnaire. 

OREA provided a questionnaire for teachers to measure this objective. The 
results indicated that 100 percent of participating staff used the project-provided 
E.S.L strategies, modified or developed their own materials to use with LEP students, 
and/or used materials modified or developed by their colleagues. 

The project met its objective for staff use and modification or development of 
E.S.L. strategies. 

• By June 30, 1993, 65 percent of the participating teachers will have 
established mechanisms for meeting regularly with E.S.L/bilingual 
and/or subject area teachers of LEP students at the same grade level, 
as measured by participating teacher and Resource Teacher logs. 

All E.S.L teachers established mechanisms for meeting regularly with 
E.S.L/bilingual and or subject area teachers of LEP students. This was 
demonstrated by logs which were kept on meetings with the classroom/bilingual 
teachers. 

The project met its objective for mechanisms established for meetings. 
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PARENTAL INVOLVEMENT OUTCOMES 

An OREA consultant observed a parent E.S.L. class at P.S. 33. The class 
began with the students saying "good morning" to each other. The resource 
specialist then began a discussion about the date and weather and put key phrases 
on the blackboard. The class discussed holidays, colors, television shows, sports, 
animals, fruits, and foods, as the resource specialist put the words on the blackboard. 
Then the resource specialist distributed a survey which asked, "What is your favorite 
holiday, color, sport, (etc.)?" Parents were to fill out the survey themselves and also 
interview a friend and fill out their answers. 

• By June 1993, at least 60 percent of the parents of participating 

students will have taken part in three or more activities for parents, as 
measured by attendance records for parental involvement activities. 

A variety of parental activities were sponsored by Project CALLA, such as 
PAC meetings, E.S.L. classes, and SABE and NAAPAE conferences. These activities 
were well attended: for example, 100 parents attended the SABE conference. The 
project coordinator indicated, however, that while great strides had been made 
toward meeting this objective, the enormous number of students served by the 
project made it unrealistic to expect 60 percent of parents to attend activities. It was 
also difficult to document whether the parents that attended these activities were the 
parents of project students. 

The project did not meet the parental involvement objective for activity 

attendance. The project director indicated that a greater emphasis will be placed on 

documenting the involvement of project parents in activities. 

29 
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• By June 1993, at least 60 percent of the parents of participating 

students will have demonstrated a satisfactory level of understanding of 
and participation in their children's education, as measured by a parent 
checklist and by teachers reports. 

OREA provided a questionnaire for parents in English, Chinese, and Spanish 

to measure this objective. Since it would be unrealistic to attempt to contact even 60 

percent of the parents of the almost 1 ,000 participating students, questionnaires were 

given to the parents of the random stratified sample of 95 students called for in the 

student objectives. 

Based on the questionnaires returned, 89 percent of parents indicated that 
they felt that they had a better understanding of their child's education, and 96 
percent indicated that they participated more in their child's education as a result of 
the project. 

The project met its objective for parental understanding and participation. 
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IV. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 

ACHIEVEMENT OF OBJECTIVES 

The project met its student objectives for English language development and 
the content areas, as well as all its staff development objectives. The project met one 
of two parental involvement objectives; the objective for parental participation in 
activities was not met. This may have been due to inadequate records, or the 
unrealistically high percentage of parent participation called for in the objective. The 
project director indicated that a greater emphasis will be placed on documenting the 
involvement of project parents in activities. 

The project not only benefited the students academically but also increased 
awareness of their own and other cultures. Teachers were given training by the 
project resource specialist. Parents were provided with E.S.L. classes as well as 
other activities. 

MOST AND LEAST EFFECTIVE COMPONENTS 

The project coordinator believed that the training for teachers was particularly 
effective, and reactions to it were positive. In addition, parental E.S.L classes were 
well received. The coordinator also felt that it would be beneficial to hire additional 
resource teachers. There was only one resource teacher (who also served as project 
coordinator) and one neighborhood family worker to provide training and support to 
the ten CALLA schools, 44 participating teachers, and almost 1,000 students. 
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RECOMMENDATION TO ENHANCE PRO JECT EFFECTIVENESS 

The following recommendation was made based on the findings of this 

evaluation: 

• Seek to modify the objective for parental participation in activities, to 
make it more realistic. 
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APPENDIX A 



Instructional Materials 

Scholastic Publications 

How the Forest Grew 
Animals on the Job 
Backyard Insects 
Antarctica 

Experiment with Water 
Fossils Tell of Long Ago 
Nine True Dolphin Stories 
A Picture Book of Hanukkah 
A Seed Is a Promise 
Sharks 

Skeletons! Skeletons! 
Storms 

Sun, Stars, and Planets 
Water's Way 

Weather Words and What They Mean 

Wild Animal Families 

Wonders of Science 

Fabulous Facts About the 50 States 

Facts and Fun About the Presidents 

50 Simple Things Kids Can Do to Save the Earth 

Amazing Earth Adventures 

Buildings, Bridges, and Tunnels 

The Call of the Wolves 

The Cloud Book 

Chickens Aren't the Only Ones 

The Long Way to a New Land 

Frederick Douglass Fights for Freedom 

The Story of George Washington Carver 

True Stories About Abraham Lincoln 

What Are You Figuring Now? 

It's Groundhog Day 

It's April Fools Day 

The Pilgrim's First Thanksgiving 

Flags of the United Nations 

Paper Tricks 

The Photography Book 

Harry Houdini 

The First Woman Doctor 
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They Led the Way 

The Story of Thomas Alva Edison, Inventor 
The Kids No-Cook Cookbook 
Chicken Soup With Rice 

Alexander and the Terrible, Horrible, No Good, Very Bad Day 
A Picture Book of Martin Luther King 
Arroz con leche 

The Three Billy Goats Gruff r 

The Twelve Days of Christmas 

Las Navidades 

The Night After Christmas 

The Night Before Christmas 

The Chanukkah Guest 

Peter Cottontail's Easter Book 

Eating Fractions 

Jesse Bear What Will You Wear? 
Play Rhymes 

I Am Curious About Seasons 
Raintree/Steck-Vauahn Publishers 

First Starts (set) 

Level C Our Communities 

America's Story 

Modern Curriculum Press 

Multicultural Celebrations I (set) 

Exploring Our Work, Level A (set) 

Tu Salud tu mundo 

Cuantas forms 

The Universe 

All of a Kind Family 

El Paseo de Niki 

La Banca en el Parque 

El Cocodrilo Con Mucha Hambre " 
Una Tormenta de Verena 
Poetry Power E.S.L Set 
Our Solar System 

Riqbv 

What's In the Cupboard 
Green Bananas 
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Creature Feature 

Off to Work 

A Farm's Not a Farm 

When the Circus Comes to Town 

Animal Clues 

Save Our Earth 

Book of Animal Records 

The Lost Dinosaur 

The Three Little Pigs 

Penny Penguin's Party 

Shiny Shells 

A Strange Stew 

Ten Silly Sheep 

A Mean Machine 

Paint A Rainbow 

Our Sister's Surprise 

Mr. McMunch 

Pirates' Gold 

The Great Carrot Mystery 

Sundance 

Reading Math Themes 
Reading Science Themes 
Reading Social Studies Themes 
Strega Nona 
The Day of the Rainbow 
The Magic Wings 
Destination: Antarctica 
Hands 

Mei Ling's Tiger 
I Have A Dream 

The Alphabet Book/El Abecedario 
Tortilla For Emily 

Addison-Weslev 

Complete Program-Multicultural Sing-Along Big Book Program 
Complete Program-Early Childhood 
The Gingerbread Man 

Attanasio and Associates. Inc. 

Jazz Chant Fairy Tales Kit 
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Thft Wri ght Group 



Twig Books 

First Nature Watch 

My Wonderful Aunt Story 1 , 2 

Now I Am Six 

You are Special 

Fact and Fantasy 

Too Big For Me 

Mrs. Wishy-Washy 

Grandpa, Grandpa 

The Hungry Giant 

Dan the Flying Man 

Go, Go, Go 

Shopping Set 

Baby Gets Dressed. 

Sunshine Read Together Big Books 

I Dream 

Shopping Bag Set 

Baby Gets Dressed Set A 

The Birthday Cake Set A 

Huggles Breakfast Set A 

Our Granddad Set B 

The Long, Long Tail Set B 

What is a Huggles? Set B 

Yuck Soup Set B 

My Home Set C 

When Itchy Witchy Sneezes 

Uncle Buncle's House Set C 

What Would You Like? Set E 

Un abrazo es tan caluroso Set D 

Mi hogar Set C 

Yo amo a mi familia Set C 

Nuestra calle Set D 

Environments. Inc. 

Rain Forest 

Will I Have a Friend? 

Quiet Mother and Noisy Little Boy 

How My Parents Learned to Eat 

The Circus Baby 

The Carrot Seed 

Monarch Butterfly 

From Seed to Plant 
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APPENDIX C 
Questionnaires 



BILINGUAL, MULllf^TURAL* AND EARLY CH1UDHOOO EVALUATION 

OFFICE OF RE^8&H, % EVALUATlON. AND ASSESSMENT 

NEW YORK CITY PUBLIC SCHOOLS 

UO LIVINGSTON STREET. ROOM 732 

BROOKLYN. NY 11201 

(718)935-3790 FAX (718)935-5490 



******* 



STAFF DEVELOPMENT ASSESSMENT 



TTOIT 

1 2 3 



Program: CALLA 



In the boxes at the right, mark 11 Y" for Yes, "N" for No. 



As a result of participating in the Summer Institute of pre- 
service training for Project CALLA: 



1. Did you increase your knowledge of the content of 
cognitive approaches to English language development? 

2. Did your knowledge of the methodology of cognitive 
approaches to English language development increase? 



g 



Thank you very much* 



4,9 



/biu> guau multicultural, akd 

' OFFICE OF RESEARCH ISVAJLUATIC 1 
NEW YORK Cm PUBLIC SCHOOLS 
t iO UVWOSTON STREET, ROOM TJX 
BROOKLYN, NY U»t 
(7U)93M790 FAX (71$) 935-5490 




,Y CHILDHOOD EVALUATION 
ASSESSMENT 



STAFF DEVELOPMENT QUESTIONNAIRE 



Program: CALLA 



Directions : Please write *Y" for Yes, "N H for No in the boxes at the right 



1. As a result of participating in Project CALLA, have you used the 
E.S.L. strategies provided by the project? 

n 

3 

2. Have you developed and/or modified your own materials to use with 
LEP students? 

□ 

4 

3. Have you used E.S.L. materials developed and/or modified by other 
staff members in your classroom? 

□ 



THANK YOU. 
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WUNGUAL* MULTICULTURAL, AND EAALY CHIlI>HOOO EVALUATION 

OmCBOT RESEARCH EVALUATION, AMD ASSESSMENT 

NEW YOWC Cm fUBUC SCHOOL! 

UO UVTNOJTOH mUOT, ROOM t» 

BROOKLYN, NY tt*l 

(711) fSS-mO PAX (711) ns-$«o 




PARENT QUESTION NA1RE/CUESTI0NARI0 PARA PADRES/ 

SB 



Program: CALLA 



I 2 



Directions : Please write *Y* for Yes and *N* for No in the boxes at the right 



Since your child has participated in Project CALLA: 

1. Do you feel that you have a better understanding of his/her education? 

2. Have you participated more in his/her education? 



□ 

3 

□ 



Programa: CALLA 



Instmcciones : Por favor escriba en el recuadro a la derecha - S* por SI y "N* por No. 



Desde que su hijo est* participando en el Proyecto CALLA: 
1. iPiensa usted que entiende mejor su educacidn? 



2. 6Ha aumentado su participaci6n en la misma? 



□ 

3 

□ 
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□ 

□ 



